Tactile feedback mechanism for a data processing system

ABSTRACT

An apparatus for providing a tactile stimulus to a part of the body of a physical operator when a virtual operator, created by movements of the physical operator, encounters a virtual object defined by a computer. A signalling unit communicates with the computer and emits a signal when the virtual operator encounters a virtual object. A stimulus unit responsive to the signalling unit is disposed in close proximity to a part of the body of the physical operator for providing a tactile stimulus when the virtual operator encounters a virtual object. The stimulus unit may comprise a segment of memory metal which undergoes a martensitic transformation to a different form or a solenoid having a member which moves in response to a signal emitted by the signalling unit. A vibrating member, such as a piezoceramic bender may be used instead of or in addition to the solenoid or memory metal.

This application is a continuation of U.S. patent application Ser. No.08/335,302, entitled “Tactile Feedback Mechanism for a Data ProcessingSystem”, Harvill, et al., filed Oct. 27, 1992, now U.S. Pat. No.5,986,643 which was a continuation of U.S. patent application Ser. No.07/746,292, entitled “Tactile Feedback Mechanism for a Data ProcessingSystem”, Harvill, et al., filed Aug. 13, 1991 now abandoned, which was acontinuation of U.S. patent application Ser. No. 07/315,252, entitled“Tactile Feedback Mechanism for a Data Processing System”, Harvill, etal., filed Feb. 21, 1989 now abandoned, which was a continuation of U.S.patent application Ser. No. 07/029,930, entitled “Tactile FeedbackMechanism for a Data Processing System”, Harvill, et al., filed Mar. 24,1987 now abandoned.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an apparatus for manipulating virtual objectsdefined by a computer based on movement of a part of a body of aphysical operator and, more particularly, to an apparatus for providinga tactile stimulus to a part of the body of the physical operator when avirtual operator, created from movements of the physical operator,encounters the virtual object.

2. Description of the Relevant Art

Many input devices are used to manipulate virtual objects represented bya computer system. For example, keyboards, joy sticks, mice, trackballs, and light pens often are used to position a cursor at a desiredlocation on a video screen so that operations may be selected withoutthe necessity of keying in multiple commands on a keyboard. However,mastering the operation of such input device is often difficult becausethe hand movements required to operate the devices do not correspond tothe visual feedback presented by the display screen of the computer.

One method and apparatus which overcomes the foregoing disadvantage isdescribed in U.S. Pat. No. 4,988,891 entitled “Computer Data Entry AndManipulation Apparatus And Method.”That invention is directed to anapparatus for converting gestures and positions of the hand of aphysical operator into a virtual operator, such as an animated versionof the hand, for manipulating virtual objects defined by the computer.Because one object of such a system is to provide a realistic simulationof virtual object manipulation by a physical operator, it would behelpful if the physical operator was provided with tactile feedback sothat the operator would have a more reliable and realistic indication ofwhen the virtual operator is deemed to have encountered the virtualobject.

SUMMARY OF THE INVENTION

The present invention is an apparatus for providing a tactile stimulusto a part of the body of a physical operator when a virtual operator,created by movements of the physical operator, encounters a virtualobject defined by a computer. In one embodiment of he present invention,a signalling unit communicates with the computer and emits a signal whenthe virtual operator encounters a virtual object. A stimulus unitresponsive to the signalling unit is disposed in close proximity to apart of the body of the physical operator for providing a tactilestimulus to the physical operator when the virtual operator encountersthe virtual object. Such tactile feedback not only provides realism, butit allows the operator to sense when the virtual object has been graspedadequately and correctly by the virtual operator for performing delicatefunctions.

The stimulus means in the embodiment comprises a segment of memory metaldisposed in close proximity to a part of the body of the physicaloperator (e.g., wrapped around a finger of the physical operator). Whenthe virtual operator encounters the virtual object, a transistorconnected to the memory metal and to the signalling unit flows a currentthrough the memory metal sufficient to raise the temperature of thememory metal so that the memory metal exhibits a shape memory effect andundergoes a martensitic transformation to a different form, whereby themovement of the metal into the different form imparts a tactile stimulusto the physical operator.

In another embodiment of the invention, a solenoid having a member whichmoves in response to the signal emitted by the signalling unit isdisposed in close proximity to a part of the body of the physicaloperator, and the moving member provides a tactile stimulus to thephysical operator whenever the virtual operator encounters the virtualobject. For example, a coil may be wrapped around a finger of theoperator, and magnets disposed between the coil and the finger vibratein response to a periodic signal emitted by the signaling unit.Alternatively, a vibrating member, such as a piezoceramic bender, may beplaced in close proximity to a part of the body of the physical operator(e.g., the fingertips of the operator's hand) for vibrating in responseto the periodic signal emitted by the signaling unit.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an overall perspective view of a data processing systemaccording to the present invention wherein movements of a part of a bodyof a physical operator are converted into a virtual operator formanipulating a virtual object represented within the data processingsystem.

FIG. 2 is a block diagram of a tactile stimulus mechanism according tothe present invention.

FIG. 3 is a palm-side view of an instrumented glove assemblyillustrating a memory metal tactile stimulus mechanism according to thepresent invention.

FIG. 4 is a back view of the instrumented glove assembly of FIG. 3.

FIG. 5 is a schematic diagram of the stimulus circuit used in theembodiment of FIGS. 3 and 4.

FIG. 6 is a sectional detail view of one finger of another embodiment ofan instrumented glove assembly according to the present inventionillustrating a solenoid unit for providing a tactile stimulus to thefinger of the operator.

FIG. 7 is a view taken along line 7—7 of FIG. 6.

FIG. 8 is a schematic diagram of the stimulus circuit used in theembodiment of FIGS. 6 and 7.

FIG. 9 is a palm-side view of an instrumented glove assembly accordingto the present invention incorporating a piezoceramic bender forproviding a tactile stimulus to the fingers of an operator.

FIG. 10 is a schematic diagram of the stimulus circuit used in theembodiment of FIG. 9.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is an overall perspective view of a data processing system 4wherein movements of a part of a physical body 10 of an operator 12 areconverted into a virtual operator 16 for manipulating a virtual object20 represented within data processing system 4. The movements ofphysical operator 12 preferably are converted into virtual operator 16through instrumentation disposed in close proximity to a part of thebody of physical operator 12, preferably on the clothing of physicaloperator 12. For purposes of illustration, the movements of a hand 22 ofphysical operator 12 are converted into virtual operator 16 through aglove 24 instrumented as described in U.S. Pat. No. 4,988,891 entitled“Computer Data Entry And Manipulation Apparatus And Method.”It is to beunderstood, however, that the present invention may be employed with anypart of the body of physical operator 12 which may be used formanipulating virtual objects defined by data processing system 4.

FIG. 2 is a block diagram of a tactile stimulus mechanism according tothe present invention. As shown therein, a programmable interval timer28, readily available as part No. 8253 from Intel Corporation, receivesdata from a CPU 32 of data processing system 4 through a bus 36.Internal timer 28 provides a signal to a stimulus circuit 40 through abias 44 whenever virtual operator 16 encounters virtual object 20.

FIGS. 3 and 4 illustrate one embodiment of stimulus circuit 40 disposedwithin a layered fabric glove 24. For ease of illustration, the outerlayer fabric of glove 24 has been removed to expose the components ofstimulus circuit 40. Encircling the tips of fingers 48 and thumb 50 ofhand 22 are tactile feedback devices 52 comprising a plurality of loops54 of a metal which exhibits a shape memory effect (hereinafter referredto as memory metal). With memory metals, a part deformed or stretched atone temperature will, upon being heated to a second temperature,completely recover its original shape. Through appropriate heattreatment, the part can be made to spontaneously change from one shapeto the other when cycled between two temperatures. In the process, themoving metal delivers substantial force. Such metals are commonly formedas nickel titanium alloys and are readily available by the trade name“NITINOL.” Each tactile feedback device 52 is connected to intervaltimer 28 through conductive wires 62, which together form bus 44 (FIG.2).

FIG. 5 is a schematic diagram of stimulus circuit 40 used in theembodiment depicted in FIGS. 3 and 4. Stimulus circuit 40 includes aDarlington transistor 66 having its base connected for receiving signalsfrom programmable interval timer 28 (FIG. 2) over bus 44 through aninverter 70. Timer 28 in this embodiment preferably is programmed foremitting a one-shot pulse whenever virtual operator 16 encountersvirtual object 20. This minimizes the risk of overheating the memorymetal. The collector of transistor 66 is connected to a currentdistribution strip 74 which is connected to each loop 54 of tactilefeedback device 52, and the emitter of transistor 66 is referenced to aground potential. Each loop 54 of tactile feedback device 52 is furtherconnected to a current ground strip 78 which is separated from currentdistribution strip 74 by an insulator 80. Current ground strip 78 isconnected to a ground potential through a fuse 82 and variable resistor86. Accordingly, when a signal is received on the base of transistor 66from bus 44, indicating that virtual operator 16 has encountered virtualoperator 20, current flows simultaneously through each loop 54 oftactile feedback device 52. Loops 54 are heated a sufficient temperatureso that they substantially simultaneously undergo a martenistictransformation to a different form (e.g., from round to oval loops),whereby the movement of the metal into the different form imparts atactile stimulus to the physical operator.

FIGS. 6, 7, and 8 illustrate another embodiment of stimulus circuit 40A,wherein a solenoid unit 90 is disposed on a finger 48 of the physicaloperator. As shown in FIGS. 6 and 7, solenoid 90 comprises a coil 94encircling finger 48 and a plurality of magnets 98A disposed betweencoil 94 and finger 48, preferably along the underside of finger 48 asshown in FIG. 7. As shown in FIG. 8, Darlington transistor 66 has itsbase connected to bus 44 through inverter 70 as before, except thatprogrammable interval timer 28 preferably is programmed for emitting aperiodic signal, such as a square wave, whenever virtual operator 16encounters virtual object 20. The collector of transistor 66 isconnected to coil 94 of solenoid 90, and coil 94 is connected to aground potential through fuse 82 and variable resistor 86. Accordingly,coil 94 receives an intermittent signal from transistor 66 and causesmagnets 98A to vibrate for providing a tactile stimulus to finger 48whenever virtual operator 16 encounters virtual object 20.

FIGS. 9 and 10 illustrate another embodiment of stimulus circuit 40Bwhich incorporates the use of vibrating members 98B, such aspiezoceramic benders, disposed on the underside of fingers 48 and thumb50. As shown in FIG. 10, stimulus circuit 40B in this embodimentincludes an operational amplifier 100 having its inverting inputconnected to bus 44 for receiving periodic signals from programmableinterval timer 28. The non-inverting input 104 of operational amplifier100 is connected to a voltage divider network consisting of resistors106 and 108. When operational amplifier 100 receives a periodic signalthrough bus 44, it provides a periodic signal sufficient to drivevibrating member 98B to provide a tactile stimulus to fingers 48 andthumb 50 whenever virtual operator 16 encounters virtual object 20.

While the above is a complete description of a preferred embodiment ofthe present invention, various modifications are obvious to thoseskilled in the art. For example, memory metal sensors 54 may be formedin any desired shape which allows the shape memory effect to impart atactile stimulus to fingers 48 and thumb 50, and solenoid 90 maycomprise a conventional solenoid having a plunger which projects againstthe surface of the skin of the physical operator for providing thetactile stimulus. The stimulus circuitry described herein may bedisposed anywhere in close proximity to a part of the body of thephysical operator, and the part of the body stimulated need notcorrespond to the part of the body which is converted into the virtualoperator. Each sensor 54, solenoic 90, or vibrating member 98B may beconnected independently so that different fingers on virtual operator 16activate different sensors, depending on which virtual finger encountersvirtual object 20. Consequently, the description should not be used tolimit the scope of the invention, which is properly described in theclaims.

1. An apparatus for providing a stimulus to a human operator, theapparatus for use in a virtually reality system in which a part of thebody of the human operator is depicted as a virtual body part in avisual display, the apparatus comprising: a position sensing subsystemconfigured to track a position of the part of the body of the humanoperator; a signaling unit coupled to the position sensing subsystem andconfigured to provide an indication in response to the virtual body partcontacting a virtual object; and a vibrating member for providing atactile stimulus as the part of the body of the human operator inresponse to the indication from the signaling unit.
 2. The apparatus asrecited in claim 1, wherein the vibrating member comprises apiezoceramic bender.
 3. The apparatus as recited in claim 2, furthercomprising a glove to be worn by the human operator, wherein thepiezoelectric bender is attached to the glove.
 4. The apparatus asrecited in claim 2, further comprising a glove to be worn by the humanoperator, wherein the glove includes an inner layer and an outer layer,and wherein at least a portion of the piezoceramic bender is disposedbetween the inner layer and the outer layer of the glove.
 5. Theapparatus as recited in claim 2, wherein the piezoceramic bender isdisposed in close proximity to a part of the body of the human operator;and wherein the piezoceramic bender is coupled to a signal generator,wherein the signal generator is operable to produce, depending upon theindication from the signaling unit, a signal that causes thepiezoceramic bender to vibrate, whereby a vibration of the piezoceramicbender imparts a tactile stimulus to the human operator.
 6. Theapparatus as recited in claim 5, wherein the signal generator comprisesan operational amplifier coupled to selectively control a flow ofcurrent through the piezoceramic bender depending upon the indicationfrom the signaling unit, and wherein the signal generator provides aperiodic signal to the operational amplifier when the virtual operatorcontacts the virtual object.
 7. The apparatus as recited in claim 5,wherein the piezoceramic bender is positioned on a fabric to be placedin close proximity to a part of the body of the human operator.
 8. Theapparatus as recited in claim 7, wherein the piezoceramic bender ispositioned on the fabric such that the piezoceramic bender couples to apart of the body of the human operator when the fabric is worn by thehuman operator.
 9. The apparatus as recited in claim 8, wherein the partof the body coupled by the piezoceramic bender is a finger.
 10. Theapparatus as recited in claim 9, wherein the piezoceramic bender isdisposed on the underside of the finger.
 11. An apparatus forinteracting with a virtual object represented within a display, theapparatus comprising: a glove adapted to be worn on a part of a user'sbody; a processing system configured to generate a virtualrepresentation of the part of the user's body on the display; a positionsensor coupled to the processing system, wherein the position sensor isconfigured to detect a position of the part of the user's body; asignaling unit configured to provide an indication in response to thevirtual representation of the part of the user's body contacting thevirtual object within the display; and a piezoceramic bender configuredto provide a tactile stimulus to the part of the user's body dependingupon the indication from the signaling unit.
 12. The apparatus asrecited in claim 11, wherein the position sensor includes a transmitterand a receiver, wherein the transmitter conveys a wireless transmissionto the receiver, and wherein the position sensor is configured todetermine the position of the glove depending upon the wirelesstransmission.
 13. The apparatus as recited in claim 12, wherein thetransmitter is an ultrasonic transmitter.
 14. The apparatus as recitedin claim 11, wherein the piezoceramic bender is disposed in closeproximity to a part of the body of the human operator, and wherein thepiezoceramic bender is coupled to a signal generator, wherein the signalgenerator is operable to produce, depending upon the indication from thesignaling unit, a signal to cause the piezoceramic bender to vibrate,whereby a vibration of the piezoceramic bender imparts a tactilestimulus to the human operator.
 15. The apparatus as recited in claim11, wherein the glove includes an inner layer and an outer layer, andwherein at least a portion of the piezoceramic bender is disposedbetween the inner layer and the outer layer of the glove.
 16. Anapparatus for interacting with a virtual object represented within adisplay, the apparatus comprising: a processing system configured togenerate a virtual representation of a part of a user's body on thedisplay; a position sensor coupled to the processing system, wherein theposition sensor is configured to detect a position of the part of theuser's body; a signaling unit configured to provide an indication inresponse to the virtual representation of the part of the user's bodycontacting the virtual object within the display; and a piezoceramicbender configured to provide a stimulus to the part of the user's bodydepending upon the indication from the signaling unit.
 17. An apparatusfor use with a virtual reality system including a display, the apparatuscomprising: a glove adapted to be worn on a body part of a user; aprocessing system configured to generate a representation of the bodypart on the display; a position sensor coupled to the processing system,wherein the position sensor is configured to detect a position of thebody part; a piezoceramic bender attached to the glove, wherein thepiezoceramic bender is configured to provide a tactile stimulus to theuser in response to the representation of the body part visuallycontacting a representation of an object depicted on the display. 18.The apparatus as recited in claim 17, further comprising a flex sensorattached to the glove, wherein the flex sensor generates a signalindicative of a flexing of the part of the user's body.
 19. A method forproviding a stimulus to a part of a body of a human operator using avirtual reality system in which the part of the body of the humanoperator is depicted as a virtual body part in a visual display, themethod comprising: sensing a position of the part of the body of thehuman operator; displaying said virtual body part in said visualdisplay; producing an indication in response to the virtual body partcontacting a virtual object; and activating a piezoceramic bender inresponse to the indication to provide a tactile stimulus to the humanoperator.
 20. The method as recited in claim 19, further comprising thehuman operator wearing a glove including the piezoceramic bender whichprovides the tactile stimulus.